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Local  Effects  and  Global  Impact  of  DEOMI  Training 

Judith  L.  Johnson,  Ph.D. 

LA  Tech  University 

Abstract 

This  report  contains  findings  from  three  studies  oriented  toward  program  evaluation  of  DEOMI 
training  effectiveness.  Study  One  was  a  resurvey  of  commander/supervisor  satisfaction  of 
graduate  performance  and  demonstrated  consistently  high  levels  of  satisfaction  across  a  one-year 
interval.  Study  Two  ascertained  commander/supervisor  initial  satisfaction  with  DEOMI  graduates 
and  found  levels  to  be  high  across  five  content  domains.  For  both  studies,  heuristic  findings 
emerged  through  use  of  a  method  designed  to  elicit  critical  commentary  regarding  DEOMI. 
Study  Three  evaluated  local  effects  of  DEOMI  training  through  pretest  and  posttest  measures  of 
achievement  and  attitudinal  change.  It  was  found  that  measures  of  achievement  were  significantly 
higher  on  completion  of  training.  However,  authoritarian  attitudes  were  not  significantly  different 
upon  conclusion  of  training.  Findings  are  discussed  and  future  research  areas  identified. 


The  opinions  expressed  in  this  report  are  those  of  the  author  and  should  not  be  construed  to 
represent  the  official  position  of  DEOMI,  the  military  services,  or  the  Department  of  Defense. 


Local  Effects  and  Global  Impact  of  DEOMI  Training 


This  report  represents  continued  effort  to  evaluate  training  effectiveness  of  the  Defense 
Equal  Opportunity  Management  Institute  (DEOMI).  For  a  review  of  the  history  of  program 
evaluation  efforts  at  DEOMI,  the  reader  is  referred  to  DEOMI  Research  Series  Publication  95-8 
entitled  "A  Preliminary  Investigation  into  DEOMI  Training  Effectiveness"  (Johnson,  1995). 

Briefly,  evaluation  of  training  effectiveness  entails  investigation  into  both  local  effects  and 
global  impact.  Local  effects  of  training  include  variables  such  as  mastery  of  course  content, 
student  attitude  change,  and  acquisition  of  skills  such  as  problem-solving  or  mediation.  Global 
impact  includes  the  effect  of  training  when  the  graduates  return  in  their  EO  capacity  to  their  home 
installations.  Ultimately,  issues  addressed  by  global  impact  include  fostering  a  productive  and 
efficient  work  climate  and  military  readiness.  It  is  important  to  note  that  selection  of  both  local 
effect  and  global  impact  variables  should  be  based  upon  the  unique  needs  of  the  organization 
(Blake  &  Heslin,  1993). 

Although  local  effects  are  useful  in  documenting  training  effectiveness  on  an  individual 
level,  global  impact  addresses  the  broader  question  of  whether  DEOMI  has  an  effect  on  the 
military  in  general.  Ideally,  assessment  of  global  impact  would  include  longitudinal  analyses  of 
EO  climate  with  appropriate  comparison  groups  (such  as  police  units).  However,  limited 
personnel  and  financial  resources  often  preclude  such  comprehensive  and  systematic  efforts. 

Consequently,  the  present  study  indirectly  assessed  global  impact  through  survey  of 
commander  and  senior  leadership  satisfaction  with  DEOMI  graduates  across  certain  domains. 
Two  separate  surveys  were  conducted  and  participants  completed  the  Commander's/Supervisor's 
Evaluation  of  DEOMI  Graduates  (CSEG)  and  Critical  Review.  Study  One  consisted  of  a 
resurvey  of  the  commanders  and  supervisors  surveyed  in  the  summer  of  1995  regarding  DEOMI 
graduate  performance.  These  individuals  received  the  same  survey  as  before  (the  CSEG).  They 
were  also  asked  to  assume  the  role  of  DEOMI's  worst  critic  and  generate  three  criticisms  of 
DEOMI  (the  Critical  Review).  This  latter  request  was  designed  to  elicit  criticism  in  that  past 
surveys  suggested  the  presence  of  positive  response  bias.  Results  were  then  compared  with  those 
from  the  summer  of  1995  in  order  to  establish  whether  satisfaction  with  DEOMI  training  was 
maintained  over  time. 

Study  Two  consisted  of  a  survey  of  the  commanders  and  supervisors  of  three  classes  of 
recent  DEOMI  graduates,  95-2,  95-3,  and  96-1.  Respondents  completed  the  CSEG  regarding 
level  of  satisfaction  with  the  DEOMI  graduates'  performance  of  EO  tasks  at  their  home 
installations.  Respondents  also  completed  the  Critical  Review. 


Study  Three  consisted  of  measurement  of  local  effects  manifested  through  two  domains: 
mastery  of  EO  content  and  degree  of  attitude  change  with  respect  to  authoritarianism.  The  class 
of  96-2  was  surveyed  with  respect  to  authoritarianism  to  assess  extent  of  change  ostensibly 
related  to  training.  A  pretest  posttest  design  was  used.  Mastery  of  EO  content  was  assessed  for 
the  classes  of  95-2,  95-3,  and  96-1. 

Mastery  of  EO  content  is  a  measure  of  achievement.  Students'  knowledge  of  EO  issues 
and  policy  is  assessed,  and  they  are  instructed  in  broad  areas  such  as  communication,  interface 
with  commanders,  and  processing  of  discrimination  complaints.  Specifically,  EO  content  also 
includes  instruction  in  identification  of  sexual  harassment  and  racism  and  provision  of  guidance  to 
personnel  who  perceive  discrimination.  Since  students  typically  would  not  possess  extensive 
experience  in  or  knowledge  of  EO  issues,  it  was  expected  there  would  be  significant  changes  in 
mastery  of  EO  content  as  a  function  of  DEOMI  training. 

Local  effects  were  also  assessed  through  measurement  of  attitudinal  change  for  the  class 
of  96-2.  Authoritarianism  was  selected  as  an  attitudinal  measure  potentially  sensitive  to  DEOMI 
training  due  to  its  established  and  well  replicated  relation  with  prejudice  (Duckitt,  1994;  Stephan, 
Ageyev,  Coates-Shrider,  Stephan,  et  al.,  1994).  Further,  this  general  relationship  extends  to  both 
traditional  prejudice  and  symbolic  racism  (Raden,  1994).  Therefore,  consistent  with  Hope's 
(1979)  findings  on  decrements  of  prejudice  as  a  function  of  DEOMI  training,  it  was  predicted  that 
authoritarian  attitudes  would  show  a  similar  decrease. 

STUDY  ONE 


Method 

Participants.  Commanders  and  supervisors  of  graduates  fi'om  DEOMI  classes  of  94-3 
and  95-1  (N  =  147)  were  resurveyed  regarding  satisfaction  with  the  graduates'  performance. 
Respondents  included  88%  male  (N  =  56)  and  12%  female  (N  =  7).  Racial/ethnic  distribution 
included  74%  Caucasian  (N  =  47)  and  19%  African-American  (N  =  12).  There  was  one  Hispanic, 
two  Asians,  one  Native  American,  and  one  "Other."  With  respect  to  rank,  52%  were  officers  (N 
=  33)  and  42%  were  enlisted  members  (N  =  27).  Respondents  included  63%  Army  (N  =  40), 
20%  Navy  (N  =  13),  9%  Air  Force  (N  =  6),  and  8%  Marines  (N  =  4). 

Procedure.  Participants  were  mailed  the  CSEG  to  complete  anonymously.  They  also 
received  a  "Critical  Review,"  which  asked  them  to  pretend  to  be  DEOMI's  "worst  critic"  and 
generate  three  major  issues  with  DEOMI.  Follow-up  procedures  resulted  in  a  return  rate  of  57% 
(N  =  63)  for  the  CSEG  as  of  the  writing  of  this  report. 
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Results 


Two  important  findings  emerged  from  this  study.  First,  commander/supervisor 
satisfaction  for  DEOMI  combined  classes  of  94-3  and  95-1  between  the  summer  of  1995  and  the 
summer  of  1996  was  not  significantly  different.  Specifically,  the  categories  of  EO  Issues  (t  =  .28, 
156  degrees  of  fi-eedom,  2  <  -77),  Command  &  Leadership  (t  =  .83,  135  degrees  of  freedom,  p  < 
.40),  Administrative,  Meetings  &  Training  (t  =  1.05,  125  degrees  of  freedom  p  <  .29),  Guidance, 
Advisement,  &  Processing  (t  =  .38,  150  degrees  of  freedom,  p  <  .70)  and  General  Satisfaction  (t 
=  .78,  121  degrees  of  freedom,  p  <  .43)  were  not  significantly  different  between  the  two  survey 
administrations.  This  indicates  relatively  stable  levels  of  satisfaction  with  DEOMI  graduate 
performance  across  a  one-year  interval. 

Secondly,  summarized  ratings  indicated  high  levels  of  satisfaction  across  categories  of 
performance  for  the  DEOMI  graduate.  Specifically,  mean  ratings  of  4.74  (SD  =  .55)  for  EO 
Issues,  4.60  (SD  =  .58)  for  Guidance,  Advisement,  &  Processing,  4.56  (SD  =  .68)  for  Command 
&  Leadership,  4.58  (SD  =  .62)  for  Administrative,  Meetings,  &  Training,  and  4.38  (SD  =  .75)  for 
General  Satisfaction  were  obtained.  Note  these  ratings  ranged  from  a  one  (1)  "Totally 
dissatisfied,"  to  a  five  (5)  "Totally  satisfied."  Results  are  summarized  in  Table  One. 

Table  1 

COMPARISON  OF 

COMMANDER/SUPERVISOR  SATISFACTION 
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Preliminary  analyses  were  performed  on  the  Critical  Review.  Although  not  subjected  to  a 
formal  content  analysis,  it  can  be  reported  that  approximately  46%  (N  =  29)  of  the  respondents 
provided  critical  review  commentaiy.  Preliminary  content  analysis  indicates  three  major  content 
areas.  The  first  area  concerned  leadership  support  of  DEOMI.  Eleven  comments  focused  on  the 
need  for  more  leadership  representation  among  DEOMI  trainees  and  greater  leadership  support  of 
DEOMI's  mission.  Secondly,  six  comments  focused  on  the  need  for  more  service-specific 
training.  Finally,  six  comments  focused  on  a  perception  that  DEOMI  graduate  performance  can 
be  viewed  by  some  individuals  as  counterproductive.  That  is,  a  focus  on  diversity  and  differences 
is  occasionally  viewed  as  potentially  less  valuable  than  a  focus  on  positive  EO  changes  made 
within  the  past  decade.  It  must  be  emphasized  these  elicited  comments  have  not  yet  been 
subjected  to  formal  content  analysis  and  should  be  viewed  as  heuristic  in  nature.  It  is  also 
important  to  note  that  many  respondents  were  unable  or  unwilling  to  generate  critical 
commentary. 


STUDY  TWO 


Method 

Participants.  Commanders  and  supervisors  for  the  combined  classes  of  95-2,  95-3,  and 
96-1  (K  =  59)  completed  the  CSEG  and  Critical  Review.  Respondents  included  86%  male  (K  = 
51)  and  13%  female  (N  =  8).  Racial/ethnic  distribution  included  67%  Caucasian  (N  =  40)  and 
18%  African-American  (N  =  11).  There  were  four  Hispanics,  two  Asians,  one  Native  American, 
and  one  "Other."  With  respect  to  rank,  61%  were  officers  (N  =  36)  and  33%  were  enlisted 
members  (K  =  20).  Respondents  included  35%  Army  (Ji  =  21),  25%  Navy  (N  =  15),  38%  Air 
Force  (N  =  23),  and  no  Marines.  The  procedure  was  the  same  as  in  Study  One.  Although  this 
study  is  presently  ongoing,  a  return  rate  of  34%  (N  =  59)  was  obtained  at  the  time  of  this  report. 

Results 

High  levels  of  satisfaction  were  obtained  for  summarized  ratings  of  EO  Issues  (Mean  = 
4.75),  Guidance,  Advisement,  &  Processing  (Mean  =  4.62),  Command  &  Leadership  (Mean  = 
4.54),  Administrative,  Meetings,  &  Training  (Mean  =  4.46),  and  General  Satisfaction  (4.25). 
Results  are  summarized  in  Table  Two. 
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Table  2 


COMMANDER'S/SUPERVISOR'S  EVALUATION 
OF  DEOMI  GRADUATES’  KNOWLEDGE  AND 
PERFORMANCE 

(N  =  59;  Combined  Classes  of  95-2,  95-3,  96-1) 
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Consistent  with  Study  One,  a  preliminary  content  analysis  was  performed  on  the  47%  (N 
=  28)  individuals  who  responded  to  this  request.  The  first  of  the  three  most  endorsed  categories 
involved  the  need  for  more  service  specific  training.  Eight  individuals  indicated  this  to  be  a  need. 
Five  individuals  reported  the  length  of  the  training  to  be  excessive.  Finally,  eight  individuals 
remarked  that  DEOMI  graduates  may  benefit  from  more  training  on  assessment  of  a  situation  in 
its  entirety  and  within  a  contextual  framework.  Some  of  these  respondents  indicated  their  belief 
that  DEOMI  graduates  need  to  be  more  flexible  and  holistic  in  their  approach  to  a  potentially 
problematic  situation.  As  in  Study  One,  it  is  emphasized  these  findings  are  preliminary. 

STUDY  THREE 


Method 

Participants.  Students  from  the  DEOMI  class  of  96-2  =  108)  were  surveyed  with 

respect  to  attitude  change.  Mastery  of  EO  Content  was  assessed  for  classes  95-2,  95-3,  and  96-1. 
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Procedure.  Students  completed  the  Authoritarianism  Scale  (AS;  Heaven,  1985)  upon 
entry  to  the  training  program.  This  inventory  has  well  established  reliability  and  validity  (Heaven, 
1985).  Students  were  retested  with  this  instrument  upon  completion  of  the  fourteenth  week  of 
the  residential  program.  Since  students  are  routinely  given  pretests  and  posttests  regarding  EO 
achievement,  information  on  content  mastery  was  readily  available  through  archival  sources. 

Results 

No  significant  differences  were  found  on  authoritarian  attitudes  between  the  pretest  and 
posttest  (t  =  .22,  degrees  of  fireedom  =  207,  p  <  .82).  The  pretest  mean  on  the  AS  was  102.83 
(SD  =  7.89)  and  posttest  mean  was  103.10  (SD  =  9.60). 

Mastery  of  EO  content  showed  significant  achievement  for  all  classes.  Results  are 
summarized  in  Table  Three.  Paired  sample  t-tests  for  the  classes  of  95-2  and  95-3  indicated 
significant  improvement  (95-2:  t  =  -27.84,  74  degrees  of  freedom,  p  <  .000;  and  95-3:  t  =  26.94, 
91  degrees  of  fireedom,  p  <  .000).  The  95-2  pretest  mean  was  56.21  (SD  =  8.83)  and  the  posttest 
mean  was  76.05  (SD  7.5).  For  95-3  the  pretest  mean  was  59.23  (SD  =  8.31)  and  the  posttest 
mean  was  76.95  (SD  =  6.21).  Paired  sample  t-tests  also  indicated  significant  improvement  in  EO 
achievement  for  the  class  of  96-1  (t  =  -22.67,  99  degrees  of  freedom,  p  <  .000).  The  class  of  96-1 
had  a  pretest  mean  of  59.57  (SD  =  8.46)  and  a  posttest  mean  of  75.55  (SD  =  5.10). 

Table  3 

EO  ACHIEVEMENT  MEANS 
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GENERAL  DISCUSSION 


Collectively,  these  three  studies  generally  demonstrate  positive  local  effects  and  global 
impact  of  DEOMI  training.  DEOMI  is  effective  in  training  as  demonstrated  through  the  increase 
in  EO  achievement  posttest  scores.  This  indicates  an  increase  in  student  proficiency  in  areas 
ranging  from  equal  opportunity  policy  through  interviewing  techniques  and  administrative 
management.  However,  as  noted  in  Johnson  (1995),  an  area  of  potential  improvement  may  lie  in 
a  focus  on  mastery  of  EO  achievement.  For  example,  a  reasonable  goal  for  mean  posttest  scores 
might  be  set  at  a  criterion  of  85%,  which  would  represent  a  higher  standard  of  achievement  than 
is  currently  attained.  Along  similar  lines,  it  is  also  useful  to  consider  an  evaluation  of  curriculum 
content  and  its  representation  in  test-taking  materials.  Students  would  benefit  from  more  frequent 
and  varied  evaluation  attempts  and  these,  in  turn,  may  more  accurately  represent  actual  course 
content. 

It  is  uncertain  why  authoritarian  attitudes  remained  unchanged.  However,  there  are 
several  possible  reasons.  First,  measurement  of  authoritarianism  is  an  indirect  assessment  of 
prejudicial  attitudes.  Future  research  efforts  should  use  instruments  directly  measuring  prejudice 
such  as  the  Modem  Racism  Scale  (McConohay,  1986).  A  second  possible  reason  is  that  the 
constmct  of  authoritarianism  and  consequent  use  of  the  Authoritarianism  Scale  may  not  be 
appropriate  for  a  militaiy  sample.  The  obtained  scores  were  high  relative  to  established  norms  for 
samples  similar  in  organizational  stmcture  (such  as  police  officers).  Given  the  unique 
organizational  structure  of  the  military,  authoritarianism  may  constitute  a  selection  factor  that 
tends  to  remain  constant  and  is  reinforced  by  service  and  the  military  hierarchy,  as  opposed  to  a 
construct  sensitive  to  training  efforts. 

With  respect  to  global  impact,  these  studies  indicate  general  conunander/supervisor 
satisfaction  with  the  DEOMI  graduate’s  performance.  Hence,  DEOMI  training  can  be 
appropriately  viewed  as  having  a  positive  impact  within  military  installations.  Importantly, 
longitudinal  study  of  commander/supervisor  satisfaction  indicates  that  levels  remained  consistently 
high  between  June  of  1995  and  June  of  1996.  This  suggests  that  the  DEOMI  classes  of  94-3  and 
95-1  maintained  high  levels  of  performance  of  their  EO  duties,  as  evaluated  by  their  supervisors. 
Hence,  performance  decrements  possibly  related  to  "burn-out"  or  decreasing  motivation  were  not 
present.  Instead,  a  high  level  of  satisfaction  with  performance  of  EO  duties  was  maintained  over 
time. 
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Graduates  received  mean  ratings  indicating  high  levels  of  satisfaction  with  management  of 
intercultural/ethnic  issues.  They  are  also  viewed  as  competent  in  managing  issues  related  to 
racism,  sexism,  and  sexual  harassment.  There  was  strong  satisfaction  expressed  regarding  the 
graduate's  ability  to  assess  EO  climate  trends  within  the  command  or  unit,  and  provision  of 
guidance  to  individuals  who  perceive  themselves  to  be  targets  of  discrimination.  Moderate 
satisfaction  was  expressed  in  performance  of  tasks  such  as  advisement  of  commanders  and  staff 
agencies  on  EO  matters  that  impact  upon  EO  climate.  Finally,  moderate  to  total  satisfaction  was 
expressed  on  conduct  of  EO  training  and  on  conduct  of  meetings,  conferences,  and  briefings 
related  to  EO  matters. 

As  noted  earlier,  commander/supervisor  ratings  for  the  classes  of  95-2,  95-3  and  96-1 
were  also  uniformly  high.  Although  caution  must  be  exercised  due  to  the  preliminary  nature  of 
this  survey,  obtained  ratings  indicate  moderate  to  high  levels  of  satisfaction  across  the  domains  of 
EO  Issues,  Guidance,  Advisement  &  Processing,  Command  &  Leadership,  Administrative, 
Meetings  &  Training,  and  General  Satisfaction. 

As  reviewed  in  Johnson  (1995),  greater  effort  could  usefully  be  directed  to  longitudinal 
trend  analysis  of  EO  climate  within  the  military.  That  is,  EO  climate  improvement  should  be 
manifest  in  both  prospective  and  retrospective  trend  analysis  of  MEOCS  data.  Although  this  may 
be  less  than  ideal,  as  addressed  earlier,  presently  limited  financial  and  personnel  resources 
preclude  more  comprehensive  efforts. 


CONCLUSIONS 

Although  findings  from  this  study  are  generally  positive  with  respect  to  local  effects  and 
global  impact  of  DEOMI  training,  there  are  several  areas  for  improving  program  evaluation 
efforts. 

First,  as  noted  in  Johnson  (1995),  a  comprehensive  theory  and  model  of  training  would 
promote  evaluation  efforts.  For  example,  if  the  training  model  and  curriculum  dictates  problem¬ 
solving  skill  acquisition  as  a  logical  goal,  then  assessment  efforts  should  be  directed  towards  this 
discrete  area.  Comprehensive  evaluation  efforts  would  thus  entail  periodic  and  systematic 
assessment  of  multiple  domains  on  multiple  occasions.  In  order  to  achieve  this,  program 
evaluation  should  be  an  ongoing  component  of  the  DEOMI  mission  and  integrated  throughout  the 
organizational  structure. 
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Secondly,  multiple  measures  and  methods  of  evaluation  would  enhance  interpretation  of 
program  evaluation  data.  For  example,  conclusions  from  the  present  study  are  constrained  by  the 
use  of  a  single  method  for  assessment  of  global  impact.  In  addition  to  survey  data  on 
commander/supervisor  satisfaction,  focus  groups  and  gathering  of  factual  information  related  to 
complaint  management  and  outcome  would  generate  much  needed  information  on  behavioral  and 
"real  life"  aspects  of  DEOMI  graduate  performance.  Focus  groups  and  other  methods  should  be 
directed  both  to  the  DEOMI  graduate  and  their  supervisor.  Ongoing  feedback  loops  could  be 
established  between  research  and  evaluation  efforts  within  DEOMI  and  field  installations.  This 
could  ensure  training  consistent  with  evolving  field  needs.  It  would  also  enable  DEOMI  training 
efforts  to  be  continually  responsive  to  dynamic  societal  and  cultural  factors  encountered  in  the  EO 
Advisor's  responsibilities,  and  ultimately  enhance  DEOMI's  capacity  to  meet  military  readiness 
needs. 
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